The Deep Q-Network (DQN) receives the most recent state of the game as input. After some preprocessing, this is the 84x84x1 (grayscale) game screen.

In order to capture temporal information, a stack of 4 preprocessed frames are used as the state that is passed to the DQN.

The DQN takes a state and action as input, and returns the corresponding predicted action value.

The DQN takes the state as input, and returns the corresponding predicted action values for each possible game action.

The DQN was provided information about the game of pong - before the agent started learning, the researchers wrote helper functions to ensure the agent always keeps the paddle close to the ball.

The DQN architecture included a combination of convolutional and recurrent layers. The CNN component captures spatial information in the state, and the RNN component detects temporal patterns.

The DQN architecture is fully convolutional: it has only convolutional layers and no fully-connected layers.

The DQN architecture is composed of a couple of convolutional layers, followed by a couple of fully connected layers.

For each Atari game, the DQN was trained from scratch on that game.

The DQN was trained on data from the first several games, and then with that knowledge (and no additional training), it could beat the remaining Atari games.